A. Field of the Invention
The present invention relates generally to methods and systems of video display or projection and, in particular, to methods, apparatus, and systems of color video signal decoding, manipulation, and display or projection with improved color gamut and brightness. Applications of the invention include, but are not limited to, video display or projection for home or commercial entertainment purposes or architectural lighting.
B. Problems in the Art
Video displays and projectors have advanced significantly in recent years. Presently, digital methods of encoding video are beginning to dominate. A variety of display and projection technologies using digital encoding have emerged.
Existing electronic video devices can be generally divided into two groups: direct view devices and projection devices. An example of a direct view device is a cathode ray tube (CRT). A direct view device will generally have a display screen composed of a plurality of RGB pixels. Each RGB pixel will be comprised of a plurality of sub-pixel elements, which most likely comprise a red sub-pixel element, a green sub-pixel element, and a blue sub-pixel element. The color image is created by the viewer's physiological and mental visual system, which mentally integrates the rapidly changing and vacillating sub-pixels and their spatial neighbors to give a full color impression.
A projection system works differently than a direct view device. A projection system displays an image by projecting light on a viewing screen or surface. Different colors of light can be created in two manners. The first method is to sequentially project separate images using only a single primary color and rapidly showing the same image with a second and then a third primary color. The images are flashed at sufficiently high frequency to give the human eye the impression that a specific color has been generated. This sequential presentation relies upon color combinations recorded by the human mind through temporal integration of the separate primary color images. An issue with the use of the sequential method is that it takes three (or the number of primary colors being used to generate the image) times the number of frames to generate the same images that could be shown with a system that integrated multiple light sources into a single image. The increased frame rate further limits the time in which a pixel can be modulated for primary color contribution control because of the fixed pixel transition time; therefore, the color resolution available decreases. The sequential system does have the benefit of having a simpler optic system. Examples of a sequential system are disclosed in U.S. Pat. Nos. 6,962,414 and 5,448,314 which are incorporated by reference in their entirety herein.
The second type of digital video projection system is a simultaneous display system based upon the concept of projecting light of all primaries (example: the red, green, blue primaries) simultaneously onto a screen. The combination is then used to generate the colors perceived by the human eye. The simultaneous (also called parallel or integrated) system integrates multiple primary light sources and provides an output comprised of an image in a single frame.
A variety of processes for digitally encoding a video signal and decoding it for display or projection have been developed and are well-known. Several approaches include using a Digital Micromirror Device (DMD) (a proprietary technology of Texas Instruments Incorporated, Dallas, Tex. USA), liquid crystal on silicon (LCoS), and liquid crystal display (LCD). Each of these technologies can be used to display an image in a sequential or parallel (integrated) color mode. Typically, these devices include a white light source separated into red, green, and blue illuminating components. The utilization of only three primary colors limits the gamut of displayable colors. The three color primaries are unable to effectively render colors throughout the entire color space the human eye can perceive.
By varying the amount of red, green, and blue light reflected from a surface or projected to a human eye, a limited number of colors in the visible spectrum, as perceived by the human eye, can be approximated.
One method of encoding three primary colors for subsequent projection of a target color is described by Lindbloom, at “RGB/XYZ Matrices.” 20 Apr. 2003. 7 Nov. 2005 http://www.brucelindbloom.com/index.html?Eqn_RGB_XYZ_Matrix.html, incorporated by reference herein. Generally, this known algorithm takes pixel data represented in a universal color coordinate system (e.g. XYZ color space), performs a color transform, and parses the result among parallel display devices or sequentially to a single device such that each pixel is represented by the combination of three primaries.
FIG. 2A generally illustrates the basic process. A conventional video processor 35 is used to encode a portion 31 of an image 30 (e.g. light blue sky from image of yellow flowers and blue sky in FIG. 2A) by representing a target color (here light blue) for a pixel 34 of the image as a combination of red (R), green (G), and blue (B). In this example of FIG. 2A, the contributions of primary red and primary green are smaller than primary blue. Thus, the output from video processor 35 would encode the target color to be displayed for that pixel as having the indicated proportional contributions from the blue, red, and green primary sources (RGB values are R=65, G=105, and B=255). Thus, the color reproduced on display device 36 would basically be as close to the light blue of the image as possible. FIG. 2A indicates diagrammatically video processor 35 could be configured to instruct sequential display of each primary (see reference #36 where the proportional contribution of blue is T=0, green at T=1, and red at T=2). Processor 35 could also operate the display device in parallel mode (e.g. red 38R, green38G, blue38B contribution simultaneous at T=0).
The purpose of this known primary color mixing algorithm is to determine the relative proportions of three given colored light sources, for example Light Emitting Diodes (LEDs), required to produce a target color.
a) CIE Color Space
The light emitted by each source is represented in CIE 2-degree 1931 format, where x and y are the chromaticity coordinates and Y′ is the photometric flux. The known quantities are represented as follows:
Primary Color Coordinates:                Given (xr,yr,Yr′), (xg,yg,Yg′), (xb,yb,Yb′)        
Target Color Coordinates:                Given (X,Y,Z)        
Although the sources are designated as red (sub r), green (sub g), and blue (sub b), it is important to note that other colors can be substituted. The three color points form the triangle 11A or gamut (see FIG. 1A) and those three coordinates, referred to as “primary colors,” can be used to create a chromaticity point that is within triangle 11A.
CIE color space is a widely used mathematical definition of color space (a color model describing the way colors can be represented by multiple color components). It was created in 1931 by the International Standard Body Commission Internationale De l'Eclairage (CIE). The CIE color space is defined by a special set of what are called imaginary primaries, which roughly correspond to red, green, and blue respectively. The chromaticity of the imaginary primaries are the XYZ tristimulus curves plotted at (0,1), (0,0), and (1,0) and are the same as the DCI 3.2.1.5 Encoding Primaries. The primaries are imaginary as they are mathematical creations, meaning their chromaticity would be plotted beyond the domain of what the human eye can perceive.
The chromaticity diagram of FIG. 1A describes CIE color space 14 in xy space. As seen in FIG. 1A, it appears as basically a closed horseshoe shape or area (see reference number 14 in FIG. 1A). The points on the border of the horseshoe (line 10), known as a spectrum locus, and the xy values corresponding to dominant wavelength excitation, range from approximately 400 nanometers to 780 nanometers, as indicated. The CIE color space 14 defines the range of color, the visible spectrum, which a typical human eye can perceive. Line 12 closes the horseshoe between the extreme monochromatic excitation after the long and short wavelengths. Region 16 of the color space, at which the human eye perceives the color “white”, lies within the closed area 14. All colors discernable by human eyes are inside the closed area 14. The closed area 14 defines the color gamut of the eye. If excitation is monochromatic, it is placed on the horseshoe border 10 and 12.
A triangle 11A with vertices R, G, and B has been superposed on color space 14 of FIG. 1A; (e.g. RGB points at REC709 standard). Vertices R, G, and B represent the position in color space 14 of primary colors Red, Green, and Blue respectively. Triangle 11A illustrates one problem with three primary color reproduction. Only hues within triangle 11A can be reproduced with those three primaries. As can be seen in FIG. 1A, the area of triangle 11A is but a fraction of the entire color space 14.
b) White Point
What is called in the art the white point, denoted by subscript w, is the color that results from the full contribution of all three sources, and may not appear white to the human eye. A proportionality constant (m) is calculated for each source and then used to calculate the (x, y) coordinates of the white point. The photometric flux of the white point is simply the sum of the flux from each source.
                                          m            r                    =                      (                                          Y                r                ′                            /                              y                r                                      )                          ⁢                                  ⁢                              m            g                    =                      (                                          Y                g                ′                            /                              y                g                                      )                          ⁢                                  ⁢                              m            b                    =                      (                                          Y                b                ′                            /                              y                b                                      )                                              [        1        ]                                                      x            w                    =                                                                      x                  r                                ⁢                                  m                  r                                            +                                                x                  g                                ⁢                                  m                  g                                            +                                                x                  b                                ⁢                                  m                  b                                                                                    m                r                            +                              m                g                            +                              m                b                                                    ⁢                                  ⁢                              y            w                    =                                                                      y                  r                                ⁢                                  m                  r                                            +                                                y                  g                                ⁢                                  m                  g                                            +                                                y                  b                                ⁢                                  m                  b                                                                                    m                r                            +                              m                g                            +                              m                b                                                    ⁢                                  ⁢                              Y            w            ′                    =                                    Y              r              ′                        +                          Y              g              ′                        +                          Y              b              ′                                                          [        2        ]            
c) Tri-Stimulus Values
The three light sources and the white point are then translated from CIE 2-degree format into XYZ tri-stimulus values (see FIG. 1A, ref no. 18) using the following equations:
                                          X            r                    =                                                                                          x                    r                                    ⁢                                      Y                    r                    ′                                                                    y                  r                                            ⁢                                                          ⁢                              Y                r                                      =                                                            Y                  r                  ′                                ⁢                                                                  ⁢                                  Z                  r                                            =                                                                    (                                          1                      -                                              x                        r                                            -                                              y                        r                                                              )                                    ⁢                                      Y                    r                    ′                                                                    y                  r                                                                    ⁢                                  ⁢                              X            g                    =                                                                                          x                    g                                    ⁢                                      Y                    g                    ′                                                                    y                  g                                            ⁢                                                          ⁢                              Y                g                                      =                                                            Y                  g                  ′                                ⁢                                                                  ⁢                                  Z                  g                                            =                                                                    (                                          1                      -                                              x                        g                                            -                                              y                        g                                                              )                                    ⁢                                      Y                    g                    ′                                                                    y                  g                                                                    ⁢                                  ⁢                              X            b                    =                                                                                          x                    b                                    ⁢                                      Y                    b                    ′                                                                    y                  b                                            ⁢                                                          ⁢                              Y                b                                      =                                                            Y                  b                  ′                                ⁢                                                                  ⁢                                  Z                  b                                            =                                                                    (                                          1                      -                                              x                        b                                            -                                              y                        b                                                              )                                    ⁢                                      Y                    b                    ′                                                                    y                  b                                                                    ⁢                                  ⁢                              X            w                    =                                                                                          x                    w                                    ⁢                                      Y                    w                    ′                                                                    y                  w                                            ⁢                                                          ⁢                              Y                w                                      =                                                            Y                  w                  ′                                ⁢                                                                  ⁢                                  Z                  w                                            =                                                                    (                                          1                      -                                              x                        w                                            -                                              y                        w                                                              )                                    ⁢                                      Y                    w                    ′                                                                    y                  w                                                                                        [        3        ]            
d) Square Matrix
The tri-stimulus values of the three light sources can then be represented as a square matrix. This matrix is inverted and multiplied by the white point to get an array of scaling factors (S). The scaling factors manipulate the matrix of the primaries in order to relate to a reference white point. If the desired white point is not the result of the full contribution of each primary color, then Sr, Sg, and/or Sb will deviate from unity. However, if the reference white is set equal to the full contribution of each primary color, then those values are equal to one.
In the event that the lighting system is being controlled by a communication protocol, which indicates the desired colors in levels of red, green, and blue, then a reference white is needed to maintain the intuitive nature that full red, green, and blue produce white. If the protocol allows the capability of selecting a specific color coordinate, then a reference white is not needed.
                              [                                                                      S                  r                                                                              S                  g                                                                              S                  b                                                              ]                =                                                                              [                                                                                                              X                          w                                                                                                                      Y                          w                                                                                                                      Z                          w                                                                                                      ]                                ⁡                                  [                                                                                                              X                          r                                                                                                                      Y                          r                                                                                                                      Z                          r                                                                                                                                                              X                          g                                                                                                                      Y                          g                                                                                                                      Z                          g                                                                                                                                                              X                          b                                                                                                                      Y                          b                                                                                                                      Z                          b                                                                                                      ]                                                            -                1                                      ⁢                                                  [            M            ]                    =                                                    [                                                                                                                              S                          r                                                ⁢                                                  X                          r                                                                                                                                                              S                          r                                                ⁢                                                  Y                          r                                                                                                                                                              S                          r                                                ⁢                                                  Z                          r                                                                                                                                                                                                  S                          g                                                ⁢                                                  X                          g                                                                                                                                                              S                          g                                                ⁢                                                  Y                          g                                                                                                                                                              S                          g                                                ⁢                                                  Z                          g                                                                                                                                                                                                  S                          b                                                ⁢                                                  X                          b                                                                                                                                                              S                          b                                                ⁢                                                  Y                          b                                                                                                                                                              S                          b                                                ⁢                                                  Z                          b                                                                                                                    ]                            ⁢                                                          [                                                                    R                                                        G                                                        B                                                              ]                        =                                                            [                                                                                    X                                                                    Y                                                                    Z                                                                              ]                                ⁡                                  [                  M                  ]                                                            -                1                                                                        [        4        ]            
e) Transform Matrix
The scaling factors are multiplied by their respective color coordinates in order to complete the transform matrix (M). The result of the matrix multiplication of the transform by the desired color coordinate [X Y Z] is an array of three values [R G B]. This array represents the relative proportions of each primary required in order to produce the desired color point.
f) Duty Cycle
These values are then scaled to the domain of [0,1] by taking each primary and dividing it by the maximum value in the primary color array.Dr=R/max(R,G,B) Dg=G/max(R,G,B) Db=B/max(R,G,B)  [5]
The result is the required duty cycle (D), which can be implemented using a number of modulation techniques (i.e. Pulse Width Modulation or Pulse Frequency Modulation). Duty cycle is the percent of power that is applied to the light source, which relates to the intensity. A duty cycle with more increments has a finer resolution, which leads to smooth transitions between colors and precision adjustments for error correction algorithms. The duty cycle calculation produces the brightest combination of the primary colors required to make the target color.
The foregoing methodology of color mixing is known in the art. It requires substantial, but manageable, mathematical calculations to operate. However, it is limited to three primary color mixing.
2. Simplified Form
These known three-color mixing calculations described above can be reduced algebraically to a simplified form that accounts for color space conversions.
If the target color is represented by (xc, yc), the relative values of the three sources required to make that color are calculated as follows:
                              color          =                      (                                          x                c                            ,                              y                c                                      )                          ⁢                                  ⁢                  R          =                      -                                                            y                  r                                ⁡                                  [                                                                                    x                        b                                            ⁡                                              (                                                                              y                            c                                                    -                                                      y                            g                                                                          )                                                              +                                                                  x                        c                                            ⁡                                              (                                                                              y                            g                                                    -                                                      y                            b                                                                          )                                                              +                                                                  x                        g                                            ⁡                                              (                                                                              y                            b                                                    -                                                      y                            c                                                                          )                                                                              ]                                                                              y                  c                                ·                                  Y                  r                  ′                                ·                                  [                                                                                    x                        b                                            ⁡                                              (                                                                              y                            g                                                    -                                                      y                            r                                                                          )                                                              +                                                                  x                        g                                            ⁡                                              (                                                                              y                            r                                                    -                                                      y                            b                                                                          )                                                              +                                                                  x                        r                                            ⁡                                              (                                                                              y                            b                                                    -                                                      y                            g                                                                          )                                                                              ]                                                                    ⁢                                  ⁢                  G          =                                                    y                g                            ⁡                              [                                                                            x                      b                                        ⁡                                          (                                                                        y                          c                                                -                                                  y                          r                                                                    )                                                        +                                                            x                      c                                        ⁡                                          (                                                                        y                          r                                                -                                                  y                          b                                                                    )                                                        +                                                            x                      r                                        ⁡                                          (                                                                        y                          b                                                -                                                  y                          c                                                                    )                                                                      ]                                                                    y                c                            ·                              Y                g                ′                            ·                              [                                                                            x                      b                                        ⁡                                          (                                                                        y                          g                                                -                                                  y                          r                                                                    )                                                        +                                                            x                      g                                        ⁡                                          (                                                                        y                          r                                                -                                                  y                          b                                                                    )                                                        +                                                            x                      r                                        ⁡                                          (                                                                        y                          b                                                -                                                  y                          g                                                                    )                                                                      ]                                                    ⁢                                  ⁢                  B          =                                                    y                b                            ⁡                              [                                                                            x                      c                                        ⁡                                          (                                                                        y                          g                                                -                                                  y                          r                                                                    )                                                        +                                                            x                      g                                        ⁡                                          (                                                                        y                          r                                                -                                                  y                          c                                                                    )                                                        +                                                            x                      r                                        ⁡                                          (                                                                        y                          c                                                -                                                  y                          g                                                                    )                                                                      ]                                                                    y                c                            ·                              Y                b                ′                            ·                              [                                                                            x                      b                                        ⁡                                          (                                                                        y                          g                                                -                                                  y                          r                                                                    )                                                        +                                                            x                      g                                        ⁡                                          (                                                                        y                          r                                                -                                                  y                          b                                                                    )                                                        +                                                            x                      r                                        ⁡                                          (                                                                        y                          b                                                -                                                  y                          g                                                                    )                                                                      ]                                                    ⁢                                  ⁢                              D            r                    =                                                    R                /                                  max                  ⁡                                      (                                          R                      ,                      G                      ,                      B                                        )                                                              ⁢                                                          ⁢                              D                g                                      =                          G              /                              max                ⁡                                  (                                      R                    ,                    G                    ,                    B                                    )                                                                    ⁢                                  ⁢                                  ⁢                              D            b                    =                      B            /                          max              ⁡                              (                                  R                  ,                  G                  ,                  B                                )                                                                        [        6        ]            
Whereas in the first three-color mixing method the target color was given in XYZ coordinates, this simplified method uses only xyY′ coordinates. The simplified equations assume that the reference white is the full contribution of each of the primary sources, so Sr, Sb, and Sg are all equal to one.
The above-described known algorithms demonstrate the reduction of a three-primary system and their duty cycles, given a target color coordinate. These algorithms have been used for color calculations of RGB LED mixing systems.
However, in a trichromat system, there is a trade-off between maximizing the color space, achieving saturated color, and being able to produce colors in the outlying areas of the CIE color space (such as pastels or deep purples).
It has been recognized that if more primary colors were added to the system, the gamut size increases and fewer performance compromises may be required. An advantage of a tetrachromat or pentachromat system (with 4 and 5 primaries, respectively) is that the spectral content of the illuminating source renders surfaces to a quality that more closely imitates natural daylight. However, one obstacle to calculating the relative contributions of systems beyond trichromat is that matrix M, discussed above, becomes non-square and is overdetermined. Its inversion, which is required to calculate the required primary contribution, is not deterministically solvable with traditional methods. Methods such as pseudoinverse using the Moore-Penrose Method could be attempted, but would provide only a least-squares fit of a sampling of the final target colors desired. The inherent approximation of the pseudoinverse calculation introduces error into an already highly complex system. The complexities stem from many sources including degradation of output with time, temperature response characteristics, and manufacturing variability. The compound effect of real-world variation and the best-fit calculations would result in the defeat of the primary objective of accurate, repeatable color.
Thus, with the three-color mixing algorithms described above, the primary values matrix cannot simply be expanded to accommodate more colors because it then becomes an unsolvable non-square matrix. In a tetrachromat or pentachromat system, the algorithm must be applied several times.
U.S. Pat. No. 6,633,302 (“the '302 patent”) recognizes the limitation on gamut utilizing only three primary sources. It purports to offer an improvement over the same. The '302 patent discloses a static display system that utilizes a cumbersome three-dimensional approach to integrating at least four primaries. This cumbersome approach represents the color space 14 in three dimensional XYZ volumes instead of two-dimensional CIE slices. In an attempt to streamline its process, the '302 patent discloses a lookup table and interpolation calculation for performing the necessary conversion from target pixel color to relative contribution of primaries. However, this approach introduces another form of estimation or approximation (including the interpolation calculation) which, in turn, introduces potential inaccuracies into the calculation of the combination of the primary sources. Thus, the '302 system is an inadequate video projection system, especially in light of the present stringent requirements of the high definition digital cinema market.
Therefore, there is room for and a real need for improvement to the state of the art. In particular, there is a need for a color mixing methodology, which allows efficient calculation of target color in color space from four or more colors.
Additionally, many of the existing technologies utilize an incandescent or gas discharge light source. As is well known in the art, such light sources have inherent variations and limitations, such as size, longevity, and durability. The broad spectrum nature of conventional light sources limits the efficiency of saturated primary production because of the filtering required. Superior display characteristics are not solely a function of accurate color reproduction, but also of the brightness and consistency of the light source. Therefore, there is also a need in the art for alternatives and improvements in light sources used for light projection.